代號說明

soft label 柔性標記 - 自 Well-Trained 模型蒸餾出有用的知識
hard label 剛性標記 - 傳統 1 或 0 表示法

Accuracy

Best Constant : predict the most frequent class

舉例

Dataset : 10 cakes, 90 eggs
Predict always eggs : accuracy = 0.9!

Logarithmic loss (logloss)

屬於 soft prediction

截圖自 Coursera

Best Constant : set αi to frequency of i-th class

舉例

Dataset : 10 cakes, 90 eggs
α = [0.1, 0.9]

Area Under Curve (AUC ROC)

pair = (red object, green object)

TP : true positive, FP : false positive
截圖自 Coursera

Best constants - All constants give same score

Cohen's Kappa motivation

屬於 hard prediction

截圖自 Coursera

舉例

Dataset : 10 cakes, 90 eggs
Baseline accuracy=0.9

predict 20 cakes and 80 eggs at random : accuracy ~0.74
0.2 * 0.1 + 0.8 * 0.9 = 0.74
error ~ 0.26

截圖自 Coursera